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THIS APPLICATION IS SUBJECT TO WITHDRAWAL FROM ISSUE AT THE INITIATIVE OF THE OFFICE OR UPON 
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I. Review the SMALL ENTITY status shown above. 

If the SMALL ENTITY is shown as YES, verify your current 
SMALL ENTITY status: 

A. If the status is the same, pay the TOTAL FEE(S) DUE shown 
above. 

B. If the status above is to be removed, check box 5b on Part B - 
Fee(s) Transmittal and pay the PUBLICATION FEE (if required) 
and twice the amount of the ISSUE FEE shown above, or 



If the SMALL ENTITY is shown as NO: 



A. Pay TOTAL FEE(S) DUE shown above, or 



B. If applicant claimed SMALL ENTITY status before, or is now 
claiming SMALL ENTITY status, check box 5a on Part B - Fee(s) 
Transmittal and pay the PUBLICATION FEE (if required) and 1/2 
the ISSUE FEE shown above. 
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of Part B - Fee(s) Transmittal should be completed and an extra copy of the form should be submitted. If an equivalent of Part B is filed, a 
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the paper as an equivalent of Part B. 

III. All communications regarding this appUcation must give the application number. Please direct all communications prior to issuance to 
Mail Stop ISSUE FEE unless advised to the contrary. 
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maintenance fees. It is patentee's responsibUity to ensure timely payment of maintenance fees when due. 



PTOL-85 (Rev. 08/07) Approved for use through 08/3 1/2010. 



Page 1 of 3 



PART B - FEE(S) TRANSMITTAL 

Complete and send this form, together with applicable fee(s), to: Mail Mail Stop ISSUE FEE 
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P.O. Box 1450 

Alexandria, Virginia 22313-1450 
or Fax (571)-273-2885 
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Determination of Patent Term Adjustment mider 35 U.S.C. 154 (b) 

(application filed on or after May 29, 2000) 

The Patent Term Adjustment to date is 1390 day(s). If the issue fee is paid on the date that is three months after the 
mailing date of this notice and the patent issues on the Tuesday before the date that is 28 weeks (six and a half 
months) after the mailing date of this notice, the Patent Term Adjustment will be 1390 day(s). 

If a Continued Prosecution Application (CPA) was filed in the above-identified application, the filing date that 
determines Patent Term Adjustment is the filing date of the most recent CPA. 

Applicant will be able to obtain more detailed information by accessing the Patent Application Information Retrieval 
(PAIR) WEB site (http://pair.uspto.gov). 

Any questions regarding the Patent Term Extension or Adjustment determination should be directed to the Office of 
Patent Legal Administration at (571)-272-7702. Questions relating to issue and publication fee payments should be 
directed to the Customer Service Center of the Office of Patent Publication at l-(888)-786-0101 or 
(571)-272-4200. 



PTOL-85 (Rev. 08/07) Approved for use through 08/3 1/2010. 



Page 3 of 3 





Application No. 


Applicant(s) 


Notice of Allowability 


10/629,403 
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Examiner 


Art Unit 
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2423 





~ The MAILING DATE of this communication appears on the cover sheet with the correspondence address- 
All claims being allowable, PROSECUTION ON THE MERITS IS (OR REMAINS) CLOSED in this application. If not included 
herewith (or previously mailed), a Notice of Allowance (PTOL-85) or other appropriate communication will be mailed in due course. THIS 
NOTICE OF ALLOWABILITY IS NOT A GRANT OF PATENT RIGHTS. This application is subject to withdrawal from issue at the initiative 
of the Office or upon petition by the applicant. See 37 CFR 1.313 and MPEP 1 308. 

1 . This communication is responsive to 7/9/2010 . 

2. ^ The allowed claim(s) is/are 1-13. 16-32.34-35 and 37-46 . 

3. □ Acknowledgment is madeof a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 

a) □ All b) □ Some* c) □ None of the: 

1 . □ Certified copies of the priority documents have been received. 

2. □ Certified copies of the priority documents have been received in Application No. . 

3. □ Copies of the certified copies of the priority documents have been received in this national stage application from the 

International Bureau (PCT Rule 17.2(a)). 
* Certified copies not received: . 

Applicant has THREE MONTHS FROM THE "MAILING DATE" of this communication to file a reply complying with the requirements 
noted below. Failure to timely comply will result in ABANDONMENT of this application. 
THIS THREE-MONTH PERIOD IS NOT EXTENDABLE. 

4. □ A SUBSTITUTE OATH OR DECLARATION must be submitted. Note the attached EXAMINER'S AMENDMENT or NOTICE OF 

INFORMAL PATENT APPLICATION (PTO-152) which gives reason(s) why the oath or declaration is deficient. 

5. □ CORRECTED DRAWINGS ( as "replacement sheets") must be submitted. 

(a) □ including changes required by the Notice of Draftsperson's Patent Drawing Review ( PTO-948) attached 

1 ) □ hereto or 2) □ to Paper No./Mail Date . 

(b) □ including changes required by the attached Examiner's Amendment / Comment or in the Office action of 

Paper No./Mail Date . 

Identifying indicia such as the application number (see 37 CFR 1.84(c)) should be written on the drawings in the front (not the back) of 
each sheet. Replacement sheet(s) should be labeled as such in the header according to 37 CFR 1.121(d). 

6. □ DEPOSIT OF and/or INFORMATION about the deposit of BIOLOGICAL MATERIAL must be submitted. Note the 

attached Examiner's comment regarding REQUIREMENT FOR THE DEPOSIT OF BIOLOGICAL MATERIAL. 



Attachment(s) 

1 . □ Notice of References Cited (PTO-892) 

2. □ Notice of Draftperson's Patent Drawing Review (PTO-948) 

3. □ Information Disclosure Statements (PTO/SB/08), 

Paper No./Mail Date 

4. □ Examiner's Comment Regarding Requirement for Deposit 

of Biological Material 



5. □ Notice of Informal Patent Application 

6. □ Interview Summary (PTO-413), 

Paper No./Mail Date . 

7. M Examiner's Amendment/Comment 

8. ^ Examiner's Statement of Reasons for Allowance 

9. □ Other . 



/Jasmine Stokely-Collins/ 
Examiner, Art Unit 2423 



/Andrew Y Koenig/ 

Supervisory Patent Examiner, Art Unit 2423 



Notice of Allowability 



Part of Paper No./Mail Date 20101020 
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EXAMINER'S AMENDMENT 

1 . An examiner's amendment to tine record appears below. Sliould tine clianges 
and/or additions be unacceptable to applicant, an amendment may be filed as provided 
by 37 CFR 1 .312. To ensure consideration of such an amendment, it MUST be 
submitted no later than the payment of the issue fee. 

Authorization for this examiner's amendment was given in a telephone interview 
with Nathan Feld on 10/20/2010. 

The application has been amended as follows: 

In claim 37, line 1 , "A computer readable memory" has been changed to -A non- 
transitory computer readable memory—. 

Allowable Subject Matter 

2. Claims 1-13, 15—32, 34-35, and 37-46 are allowed. 

3. The following is an examiner's statement of reasons for allowance: 
Regarding independent claim 1 , the prior art fails to teach or reasonably suggest 

a method for exchanging information in a shared interactive environment, comprising: 
selecting a first remote physical device in a first live video image, that shows a 
first view of the shared interactive environment, wherein information is 
associated with the first remote physical device; 

causing the information to be transferred to a second remote physical device 
shown in a second live video image that shows a second view of the shared 
interactive environment, wherein the transfer is brought about by manipulating a 
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visual representation of the information shown in the first live image by 
interacting with the first live video image and the second live video image. 

Regarding independent claim 10, tine prior art fails to teach or reasonably 
suggest a method for exchanging information in a shared interactive environment, 

comprising: 

selecting a first object wherein the first object is one of 1) a first physical device 
at a remote location shown in a live video image, and 2) an icon on a computing 
device, wherein the live video image shows a view of the remote location which 
includes a plurality of physical devices including the first physical device; 
causing information associated with the first object to be transferred to a second 
object wherein the second object is the other of 1) the first physical device at the 
remote location shown in the first live video image, and 2) the icon on the 
computing device; 

wherein the transfer is brought about by manipulating a visual representation of 
the information in the live video image; 

wherein manipulating includes interacting with the first object in the live video 
image and the second object. 

Additionally, in regards to claim 10, the prior art fails to teach or reasonably 
suggest annotating the view of the remote location shown in the live video image, 
with an annotation; 

automatically transferring the annotation to the first physical device if the 
annotation is at least partially drawn over the first physical device as it appears in 
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the live video image; and 

displaying the annotation on the first physical device such that the annotation 
can be viewed at the remote location. 

Regarding independent claim 18, the prior art fails to teach or reasonably 
suggest a method for annotating a live video image, comprising: 
displaying a live video image, wherein the live video image shows a view of a location 
including one or more physical devices at the location; 
annotating the view of the location shown in the live video image with an 
annotation; automatically transferring the annotation to one of the physical 
devices at the location shown in the live video image if the annotation is at least 
partially drawn over the physical device as it appears in the live video image; and 
displaying the annotation on the physical device such that the annotation can be 
viewed at the location. 

Regarding independent claim 24, the prior art fails to teach or reasonably 
suggest a shared interactive environment, comprising: 

a camera system to provide a first live view of a location and a second live view of the 

location, wherein the second live view can be configured to zoom in on a portion of the 
first live view, and wherein each live view shows a different view of a plurality of physical 
devices at the location; 

a first graphical user interface (GUI) coupled to the camera system which presents the 
first live view and the second live view, each view shows one or more of the plurality of 
devices at the location; 
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a device controller to dynamically control the physical device in response to interaction 
of a first user with the GUI wherein the interaction can include annotating at least one 
of: 1) the first live view of the location; and 2) the second live view of the location; 
wherein annotations are automatically transferred to the physical device in the 
live views if the annotation is at least partially drawn over the physical device as 
it appears in the live video image, and wherein the annotation is displayed on the 
physical device such that the annotation can be viewed at the location. 

Regarding independent claim 37, the prior art fails to teach or reasonably 
suggest a non-transitory computer readable memory having instructions stored thereon 
that when executed by a processor cause a system to: 

select a first remote physical device in a first live video image, that shows a first 
view of the shared interactive environment, wherein information is associated 

with the first remote physical device; and 

cause the information to be transferred to a second remote physical device 
shown in a second live video image that shows a second view of the shared 
interactive environment, wherein the transfer is brought about by manipulating a 
visual representation of the information by interacting with the first live video 
image and the second live video image. 

Regarding independent claim 46, the prior art fails to teach or reasonably 
suggest a system comprising: 

means for selecting a first remote physical device in a first live video image that 
shows a first view of the shared interactive environment, wherein information is 
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associated with the first remote physical device; and 

means for causing the information to be transferred to a second remote physical 
device shown in a second live video image that shows a second view of the 
shared interactive environment, wherein the transfer is brought about by 
manipulating a visual representation of the information by interacting with the 
first live video image and the second live video image. 

Any comments considered necessary by applicant must be submitted no later 
than the payment of the issue fee and, to avoid processing delays, should preferably 
accompany the issue fee. Such submissions should be clearly labeled "Comments on 
Statement of Reasons for Allowance." 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to JASMINE STOKELY-COLLINS whose telephone 
number is (571) 270-3459. The examiner can normally be reached on M-F 9:30-5:30 
EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 

supervisor, Andrew Koenig can be reached on (571) 272-7296. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



/Jasmine Stokely-Collins/ 
Examiner, Art Unit 2423 

/Andrew Y Koenig/ 

Supervisory Patent Examiner, Art Unit 2423 



